Skip to main content

Cluster Setup

Set up the required namespace and permissions for Kubetrace by following the steps below.

Prerequisites

Before running the onboarding script, ensure you have:
  • kubectl installed and configured
  • Access to your Kubernetes cluster
  • Admin privileges or appropriate permissions
  • Azure CLI (for Azure clusters)

Step 1: Prepare to Execute the Script

Once you have selected your Cloud provider and Cluster in the Kubetrace platform:
  1. Navigate to Step 3 of the platform
  2. Copy the script for creating a Readonly Service Account
  3. Ensure all prerequisites are satisfied before running the script

Step 2: Execute the Onboarding Script

Complete Script

#!/bin/bash
set -euo pipefail

cat <<'EOF'

                                                                      
██   ██ ██    ██ ██████  ███████ ████████ ██████   █████   ██████ ███████
██  ██  ██    ██ ██   ██ ██         ██    ██   ██ ██   ██ ██      ██
█████   ██    ██ ██████  █████      ██    ██████  ███████ ██      █████
██  ██  ██    ██ ██   ██ ██         ██    ██   ██ ██   ██ ██      ██
██   ██  ██████  ██████  ███████    ██    ██   ██ ██   ██  ██████ ███████

                                                                      
                                                                       
EOF

# ======== Input Arguments ========
if [ "$#" -ne 2 ]; then
  echo "Usage: $0 <cluster-name> <kubeconfig-output-path>"
  exit 1
fi

CLUSTER_CONTEXT="$1"
KUBECONFIG_OUTPUT="$2"
ADMIN_CONTEXT="${CLUSTER_CONTEXT}-admin"

# Try to get cluster info for resource group detection
CLUSTER_NAME="$CLUSTER_CONTEXT"
RESOURCE_GROUP=""

# Attempt to get resource group from current context
if kubectl config get-contexts -o name | grep -qx "$CLUSTER_CONTEXT"; then
  RESOURCE_GROUP=$(kubectl config view -o jsonpath="{.contexts[?(@.name=='${CLUSTER_CONTEXT}')].context.cluster}" 2>/dev/null | sed 's/.*-\([^-]*\)$/\1/' 2>/dev/null || echo "")
fi

# ======== NON-ADMIN INSTRUCTIONS FUNCTION ========
show_nonadmin_instructions() {
  echo "❌ You do NOT have AKS admin access for $CLUSTER_NAME."
  echo
  echo "📋 INSTRUCTIONS FOR ADMIN TO SET UP NON-ADMIN ACCESS:"
  echo "════════════════════════════════════════════════════════"
  echo
  echo "1️⃣ Create a Service Principal:"
  echo "   az ad sp create-for-rbac --skip-assignment --name aks-test-nonadmin -o json > nonadmin-sp.json"
  echo
  echo "   📝 This will output: { "appId": "xxx", "password": "yyy", "tenant": "zzz" }"
  echo
  echo "2️⃣ Assign AKS Cluster User Role:"
  echo "   az role assignment create \"
  echo "     --assignee <appId-from-json> \"
  echo "     --role "Azure Kubernetes Service Cluster User Role" \"
  if [ -n "$RESOURCE_GROUP" ]; then
    echo "     --scope \$(az aks show -n $CLUSTER_NAME -g $RESOURCE_GROUP --query id -o tsv)"
  else
    echo "     --scope \$(az aks show -n $CLUSTER_NAME -g <your-resource-group> --query id -o tsv)"
  fi
  echo
  echo "3️⃣ Save this YAML as 'kubetrace-rbac.yaml' and apply it:"
  echo "   ────────────────────────────────────────────────────"
  cat <<'YAML'
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: kubetrace-nonadmin
rules:
- apiGroups: [""]
  resources: ["namespaces", "serviceaccounts", "serviceaccounts/token", "secrets", "pods", "events"]
  verbs: ["get", "list", "create", "update"]
- apiGroups: ["rbac.authorization.k8s.io"]
  resources: ["clusterroles", "clusterrolebindings"]
  verbs: ["get", "list", "create"]
- apiGroups: ["apps"]
  resources: ["deployments", "replicasets"]
  verbs: ["get", "list", "create", "update"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: kubetrace-nonadmin-binding
subjects:
- kind: User
  name: "<appId-from-json>"
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: kubetrace-nonadmin
  apiGroup: rbac.authorization.k8s.io
YAML
  echo "   ────────────────────────────────────────────────────"
  echo "   kubectl apply -f kubetrace-rbac.yaml"
  echo
  echo "4️⃣ Share credentials with non-admin user:"
  echo "   📤 Send them: appId, password, tenant from step 1"
  echo
  echo "📋 INSTRUCTIONS FOR NON-ADMIN USER:"
  echo "══════════════════════════════════════"
  echo
  echo "5️⃣ Login with service principal:"
  echo "   az login --service-principal -u <appId> -p <password> --tenant <tenant>"
  echo
  echo "6️⃣ Get cluster credentials:"
  if [ -n "$RESOURCE_GROUP" ]; then
    echo "   az aks get-credentials -n $CLUSTER_NAME -g $RESOURCE_GROUP --overwrite-existing"
  else
    echo "   az aks get-credentials -n $CLUSTER_NAME -g <your-resource-group> --overwrite-existing"
  fi
  echo
  echo "7️⃣ Apply your operator.yaml file:"
  echo "   kubectl apply -f operator.yaml"
  echo
  echo "8️⃣ Then run this script again:"
  echo "   ./kubetrace.sh $CLUSTER_NAME readonly-kubeconfig.yaml"
  echo
  echo "════════════════════════════════════════════════════════"
}

# Check if admin context exists
if kubectl config get-contexts -o name | grep -qx "$ADMIN_CONTEXT"; then
  echo "✅ Admin context found: $ADMIN_CONTEXT"
  echo "🔄 Switching to admin context..."
  
  # Test if we can actually use the admin context
  if ! kubectl --context="$ADMIN_CONTEXT" auth can-i create clusterroles >/dev/null 2>&1; then
    echo "❌ ERROR: Admin context '$ADMIN_CONTEXT' exists but lacks sufficient privileges."
    echo "Unable to create cluster roles with admin context."
    echo
    if [ -n "$RESOURCE_GROUP" ]; then
      echo "Please ensure you have admin credentials by running:"
      echo "  az aks get-credentials --resource-group $RESOURCE_GROUP --name $CLUSTER_CONTEXT --admin"
    else
      echo "Please ensure you have admin credentials by running:"
      echo "  az aks get-credentials --resource-group <your-resource-group> --name $CLUSTER_CONTEXT --admin"
    fi
    echo "Then rerun this script."
    exit 1
  fi
else
  echo "⚠️  Admin context '$ADMIN_CONTEXT' not found."
  
  # Check if user has basic cluster access and can create resources
  if kubectl config get-contexts -o name | grep -qx "$CLUSTER_CONTEXT"; then
    if kubectl --context="$CLUSTER_CONTEXT" auth can-i create clusterroles >/dev/null 2>&1; then
      echo "✅ You have cluster admin privileges via regular context."
      ADMIN_CONTEXT="$CLUSTER_CONTEXT"
    else
      echo "❌ No admin privileges detected."
      show_nonadmin_instructions
      exit 1
    fi
  else
    echo "❌ ERROR: Cluster context '$CLUSTER_CONTEXT' not found."
    show_nonadmin_instructions
    exit 1
  fi
fi

# Alias kubectl with admin context for easier use
KUBECTL_ADMIN="kubectl --context=$ADMIN_CONTEXT"

# ======== Configurable Constants ========
NAMESPACE="kubetrace-readonly"
SA_NAME="kubetrace-readonly-user"
CLUSTER_ROLE="kubetrace-readonly-clusterrole"
CLUSTER_BINDING="kubetrace-readonly-binding"

# ======== Check Cluster Context Exists ========
if ! kubectl config get-contexts -o name | grep -q "^${CLUSTER_CONTEXT}$"; then
  echo " Context '$CLUSTER_CONTEXT' not found in kubeconfig"
  exit 1
fi

# ======== Create Namespace ========
echo "🔧 Creating namespace: $NAMESPACE"
if ! $KUBECTL_ADMIN get ns "$NAMESPACE" >/dev/null 2>&1; then
  $KUBECTL_ADMIN create namespace "$NAMESPACE"
fi

# ======== Create Service Account ========
echo "🔧 Creating service account: $SA_NAME"
if ! $KUBECTL_ADMIN -n "$NAMESPACE" get sa "$SA_NAME" >/dev/null 2>&1; then
  $KUBECTL_ADMIN -n "$NAMESPACE" create sa "$SA_NAME"
fi

# ======== Create ClusterRole ========
echo "🔧 Creating ClusterRole: $CLUSTER_ROLE"
$KUBECTL_ADMIN apply -f - <<EOF
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: $CLUSTER_ROLE
rules:
- apiGroups: [""]
  resources: ["pods", "services", "endpoints", "nodes", "configmaps", "secrets", "persistentvolumeclaims", "persistentvolumes", "namespaces"]
  verbs: ["get", "list", "watch"]
- apiGroups: ["apps"]
  resources: ["deployments", "replicasets", "statefulsets", "daemonsets"]
  verbs: ["get", "list", "watch"]
- apiGroups: ["batch"]
  resources: ["jobs", "cronjobs"]
  verbs: ["get", "list", "watch"]
- apiGroups: ["rbac.authorization.k8s.io"]
  resources: ["roles", "rolebindings", "clusterroles", "clusterrolebindings"]
  verbs: ["get", "list", "watch"]
- apiGroups: ["networking.k8s.io"]
  resources: ["networkpolicies", "ingresses", "ingressclasses"]
  verbs: ["get", "list", "watch"]
- apiGroups: ["storage.k8s.io"]
  resources: ["storageclasses", "csinodes", "csidrivers", "csistoragecapacities", "volumeattachments"]
  verbs: ["get", "list", "watch"]
- apiGroups: [""]
  resources: ["events"]
  verbs: ["get", "list", "watch"]
- apiGroups: ["metrics.k8s.io"]
  resources: ["nodes", "pods"]
  verbs: ["get", "list"]
- apiGroups: [""]
  resources: ["serviceaccounts/token"]
  verbs: ["create"]
  resourceNames: ["$SA_NAME"]
EOF

# ======== Create ClusterRoleBinding ========
echo "🔧 Creating ClusterRoleBinding: $CLUSTER_BINDING"
$KUBECTL_ADMIN create clusterrolebinding "$CLUSTER_BINDING" \
  --clusterrole="$CLUSTER_ROLE" \
  --serviceaccount="$NAMESPACE:$SA_NAME" \
  --dry-run=client -o yaml | $KUBECTL_ADMIN apply -f -

# ======== Generate Token (using admin context) ========
echo "🔑 Generating token for service account..."
TOKEN=$($KUBECTL_ADMIN -n "$NAMESPACE" create token "$SA_NAME")

# ======== Build Kubeconfig (using cluster info from original context) ========
echo "🧩 Building kubeconfig for readonly access..."
CLUSTER_NAME=$(kubectl config view -o jsonpath="{.contexts[?(@.name=='${CLUSTER_CONTEXT}')].context.cluster}")
API_SERVER=$(kubectl config view -o jsonpath="{.clusters[?(@.name=='${CLUSTER_NAME}')].cluster.server}")
CA_DATA=$(kubectl config view --raw -o jsonpath="{.clusters[?(@.name=='${CLUSTER_NAME}')].cluster.certificate-authority-data}")

cat > "$KUBECONFIG_OUTPUT" <<EOF
apiVersion: v1
kind: Config
clusters:
- name: $CLUSTER_NAME
  cluster:
    server: $API_SERVER
    certificate-authority-data: $CA_DATA
users:
- name: $SA_NAME
  user:
    token: $TOKEN
contexts:
- name: ${SA_NAME}-context
  context:
    cluster: $CLUSTER_NAME
    user: $SA_NAME
current-context: ${SA_NAME}-context
EOF

echo
echo "✅ Done! Read-only kubeconfig written to: $KUBECONFIG_OUTPUT"
echo "👉 Use it like:"
echo "   kubectl --kubeconfig=$KUBECONFIG_OUTPUT get pods --all-namespaces"

Example

./kubetrace-azure.sh my-aks-cluster readonly-kubeconfig.yaml

Step 3: Resources Created by the Script

When you run this script, Kubetrace automatically provisions the following components in your cluster:

Namespace

apiVersion: v1
kind: Namespace
metadata:
  name: kubetrace-readonly

Service Account

apiVersion: v1
kind: ServiceAccount
metadata:
  name: kubetrace-readonly-user
  namespace: kubetrace-readonly

ClusterRole

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: kubetrace-readonly-clusterrole
rules:
  - apiGroups: [""]
    resources: ["pods", "services", "endpoints", "nodes", "configmaps", "secrets", "persistentvolumeclaims", "persistentvolumes", "namespaces"]
    verbs: ["get", "list", "watch"]
  - apiGroups: ["apps"]
    resources: ["deployments", "replicasets", "statefulsets", "daemonsets"]
    verbs: ["get", "list", "watch"]
  - apiGroups: ["batch"]
    resources: ["jobs", "cronjobs"]
    verbs: ["get", "list", "watch"]
  - apiGroups: ["rbac.authorization.k8s.io"]
    resources: ["roles", "rolebindings", "clusterroles", "clusterrolebindings"]
    verbs: ["get", "list", "watch"]
  - apiGroups: ["networking.k8s.io"]
    resources: ["networkpolicies", "ingresses", "ingressclasses"]
    verbs: ["get", "list", "watch"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["storageclasses", "csinodes", "csidrivers", "csistoragecapacities", "volumeattachments"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["get", "list", "watch"]
  - apiGroups: ["metrics.k8s.io"]
    resources: ["nodes", "pods"]
    verbs: ["get", "list"]

ClusterRoleBinding

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: kubetrace-readonly-binding
subjects:
  - kind: ServiceAccount
    name: kubetrace-readonly-user
    namespace: kubetrace-readonly
roleRef:
  kind: ClusterRole
  name: kubetrace-readonly-clusterrole
  apiGroup: rbac.authorization.k8s.io

Step 4: Complete Setup in Kubetrace Platform

After executing the script, follow these final steps in the Kubetrace platform:
  1. Upload Kubeconfig (Step 4)
    • Copy the contents of the generated kubeconfig.yaml file
    • Paste it into Step 4 of the Kubetrace platform
  2. Add Cluster (Step 5)
    • Select the discovered cluster from the list
    • Click Add Cluster
    • Generate the Kubetrace operator configuration
  3. Deploy Operator (Step 6)
    • Copy the generated kubetrace-operator.yaml
    • Execute the following command:
      kubectl apply -f kubetrace-operator.yaml
      
  4. Verify Installation
    • Check that the operator is running properly:
      kubectl get pods -n kubetrace-readonly
      
    • All pods should be in Running status

Troubleshooting

Authorization Errors

If you encounter an error like:
(AuthorizationFailed) The client '57eada60-3d78-4de3-a599-93e8d7e4f' with object id '57eada60-3d78-4de3-a599-9db2a7e4f' does not have authorization to perform action
Solution: Follow the prerequisite instructions for admin access or least privilege setup as described in the non-admin instructions section of the script.

Admin Context Not Found

If you see ⚠️ Admin context not found, ensure you have obtained admin credentials:
az aks get-credentials --resource-group <your-resource-group> --name <cluster-name> --admin

Operator Not Running

If the operator pods are not running:
  1. Check pod logs:
    kubectl logs -n kubetrace-readonly <pod-name>
    
  2. Verify the service account has proper permissions:
    kubectl auth can-i list pods --as=system:serviceaccount:kubetrace-readonly:kubetrace-readonly-user
    

Support

For additional help or questions, please contact the Kubetrace support team or refer to the official documentation.